Guide-assisted capture of material data

ABSTRACT

A material data collection system allows capturing of material data. For example, the material data collection system may include digital image data for materials. The material data collection system may ensure that captured digital image data is properly aligned, so that material data may be easily recalled for later use, while maintaining the proper alignment for the captured digital image. The material data collection system may include using a capture guide, to provide cues on how to orient a mobile device used with the material data collection system.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/447,943, filed Jun. 20, 2019, issued as U.S. Pat. No. 10,872,426 onDec. 22, 2020, which claims the benefit of U.S. patent application62/687,741, filed Jun. 20, 2018. These applications are incorporated byreference along with all other references cited in this application.

BACKGROUND OF THE INVENTION

This invention relates to digital imaging technology and, morespecifically, a process that uses a handheld device and a capture guideto capture data of sample materials used to construct products such asclothing, jewelry, or other products.

Designers, manufacturers, and others involved in product production gettheir inspiration for products when personally handling samples ofmaterials. For example, a clothing designer may be presented with andinspired by a particular fabric they encounter in real-life, for use increating a garment they are working on. However, it is often difficultfor the designer to collaborate with others, since there is no way toeasily capture and share information about the particular fabric.

One way to do so is by keeping actual physical samples of materials in alibrary. When a particular sample is needed, the sample is found andshipped. This makes it difficult for designers to work collaboratively,especially with those whom may be at different locations.

Further, while material manufacturers may provide basic information suchas the color, pattern, and elements used to create a material, thisinformation often lacks the specificity that a designer may need in thefuture to use a material in production. Since each manufacturer may alsonot adhere to the same nomenclature as other manufacturers to describeattributes of their materials, it is difficult for the designer to view,choose, and understand how different materials may look together or howthey would work for a final product by using the manufacturer provideddescriptions alone. For example, manufacturers may classify differentblues differently than other manufactures or describe particularpatterns differently.

Some designers work with photos of materials, taken using a flatbedscanner. However, flatbed scanners are not portable, and require theuser to go to the scanner in order to capture data. A flatbed scanneralso takes poor images of materials with texture, like pebbled leather,which limits its usefulness.

Therefore, there is a need to allow accurate and useful data collectionabout materials.

BRIEF SUMMARY OF THE INVENTION

In an implementation, a material data collection system allows capturingof material data. The material data collection system allows theconversion of real-world materials in a physical space, into ahigh-quality digital asset in the virtual space that may be searched,shared, and used by others. For example, the material data collectionsystem may include digital image data for materials. The material may beany type of material whose information is to be stored in the materialdata collection system, such as fabric, knits, weaves, skins or hides,vinyl, plastics, composites, textiles, cotton, flax, wool, ramie, silk,denim, nylon, polyester, or many other materials. The material datacollection system may ensure that captured digital image data isproperly aligned, so that material data may be easily recalled for lateruse, while maintaining the proper alignment for the captured digitalimage. The material data collection system may include using a captureguide, to provide cues on how to orient a mobile device used with thematerial data collection system. An application installed on the mobiledevice may offer additional options to properly orient digital imagedata captured by the mobile device. This may provide various benefits.For example, mobile devices may vary from other based on its model,manufacturer, defects found in its parts, or other factors. This mayresult in aberrations that may affect digital image data captured by thematerial data collection system, such as a camera lens defect thatproduces as a pin-cushion effect, a fisheye effect, or other types ofaberrations. The material data collection system may transform digitalimage data, to remove or reduce these undesirable effects.

In another implementation, the material data collection system mayassist a user to correctly hold or position a mobile device whencapturing a digital image of a sample material. For example, beforecapturing the digital image, the mobile device may capture theorientation of a tabletop or other surface that the sample material isplaced upon. When the digital image is being captured, the material datacollection system may indicate to the user when the orientation of thecamera is the same or similar to the orientation of the tabletop. Theorientation of the camera is the same or similar to the orientation ofthe table top when, as measured by a accelerometer, gyroscope, or othersimilar device, the difference in the tilt, yaw, or roll of the cameraand table top is less than a predefined limit (e.g., 1 degree, 2degrees, 3 degrees, 5 degrees, or any other number of degrees).

In various embodiments, the material data collection system may includelighting information with a captured digital image. For example, onearea of a material sample may be brighter than another, because of alight source that is casting light more on one area of the materialsample than another. A capture guide included in a captured digitalimage may include one or more lighting calibration areas. Depending ondifferences between the one or more lighting calibration areas, thematerial data collection system may determine how the lighting in thecaptured digital image differs across the captured digital image. Anexample of a technique that may be used include histogram equalizationor wavelength normalization.

The material data collection system may also allow users to savemetadata with a captured digital image. For example, a user may want tonote with a captured digital image a material name, code, orientation,and size. Instead of making notes on paper or using a separatecomputerized system, the user may enter this information into thematerial data collection system and the information is associated withthe captured digital image and uploaded to a product lifecyclemanagement system.

In an implementation, the material data collection system includes usingan application executing on a mobile device to capture an initialmaterial sample image. The initial material sample image may be adigital image that includes a sample material area and a capture guidearea including a capture guide and a first alignment indicator includedon the capture guide. The capture guide area may completely enclose atleast a portion of the sample material area. The application maydetermine a first location of the initial material sample image wherethe first alignment indicator has been captured in the initial materialsample image and stores this information as a first matched alignmentindicator pair, where the first matched alignment indicator pairassociates the first alignment indicator with the first position. Theapplication transforms the initial material sample image, based on thefirst matched alignment indicator pair, to render an alignment correctedmaterial sample image. Additionally, the application identifies, fromthe alignment corrected material sample image, a sample material swatcharea, where the sample material swatch area includes at least oneinstance of a pattern found in the initial material sample image andstores on the mobile device the sample material swatch area and theinitial material sample image.

In an implementation, the material data collection system allowscorrecting positions of application provided indicators by a user. Forexample, the material data collection system may determine initial or afirst position for an alignment indicator but allow a user to change theposition of the alignment indicator. This includes displaying on themobile device the initial material sample image overlaid with a firstapplication provided alignment indicator at a first position. Forexample, there may include a base image layer with the initial materialsample image. An application generated layer, including the firstalignment indicator, may be overlaid over the base image layer. Whiledisplaying the initial material sample image, the material datacollection system may receive a first adjustment of the firstapplication provided alignment indicator from the first position to asecond position to associate the first application provided alignmentindicator with the first alignment indicator shown on the initialmaterial sample image to create a first matched alignment indicatorpair. For example, the first matched alignment indicator pair associatesthe first alignment indicator with the first application providedalignment indicator at the second position and not the first position.The association may be made by dragging the first application providedalignment indicator to the second position where, according to the user,the first alignment indicator of the capture guide appears on the baseimage layer. The first application provided alignment indicator at thesecond position may be displayed on the mobile device.

The material data collection system may include transforming the initialmaterial sample image, based on the first matched alignment indicatorpair, to render an alignment corrected material sample image. Forexample, while displaying on the mobile device the initial materialsample image overlaid with the first application provided alignmentindicator at the first position may also include a second applicationprovided alignment indicator at a third position. The material datacollection system may receive a second adjustment of the secondapplication provided alignment indicator from the third position to afourth position to associate the second application provided alignmentindicator with the second alignment indicator shown on the initialmaterial sample image to create a second matched alignment indicatorpair. The material data collection system may transform the initialmaterial sample image based upon a straight line determined by the firstmatched alignment indicator pair and the second matched alignmentindicator pair, to render an alignment corrected material sample image.The material data collection system may also include identifying, fromthe alignment corrected material sample image, a sample material swatcharea, where the sample material swatch area includes at least oneinstance of a pattern found in the initial material sample image andstoring on the mobile device the sample material swatch area and theinitial material sample image. The initial material sample image mayalso be uploaded to a product lifecycle management software for laterretrieval and use.

The material data collection system may also include transforming theinitial material sample image by compensating, based on the first matchalignment indicator pair, for at least a lens aberration introduced by alens of the mobile device when capturing the initial material sampleimage. The first adjustment may include selecting the first applicationprovided alignment indicator on a touch screen of the mobile device andmoving the first application provided alignment indicator from the firstposition to the second position. A check may be performed by thematerial data collection system to determine that the first applicationprovided alignment indicator is associated with the first alignmentindicator shown on the initial material sample image. For example, eachapplication provided alignment indicator may be associated to only onealignment guide of the capture guide.

In an implementation, the material data collection system may includeonly a portion of the initial material sample image. This may be due todisplay size limitations of the mobile device or correction by thematerial data collection system to optimize display of the capture guideand the sample material within a cut out area of the capture guide. Thecut-out portion may be of any size and shape. For example, the cut-outshape may form a bounded polygon or a partially bounded polygon. Thebounded polygon may be identified by the material data collection systemwithout a user providing input as to where the bounded polygon is shownin the initial material sample image. For example, a user is notrequired to identify lines or points that define where the boundedpolygon is shown in the image.

In another implementation, the material data collection system includesbefore capturing the initial material sample image, taking anorientation of a sample material and capture guide by the mobile device.For example, the mobile device may be laid upon a tabletop or othersurface that the capture guide is resting upon to record the orientationof the tabletop. Before capturing the initial material sample image, thematerial data collection system includes providing visual feedbackindicating whether at a given time the mobile device is in the sameorientation of the sample material and capture guide. Some examples ofvisual indication may include a number of degrees or a direction to tiltthe mobile device in order to orient the mobile device in a similarorientation as the orientation of the sample material. An accelerometerof the mobile device may be used to capture orientation information. Themethod may also include before capturing the initial material sampleimage, taking an orientation of a sample material and capture guide bythe mobile device; before capturing the initial material sample imageand after a user input to capture the initial material sample image,accessing an accelerometer of the mobile device to determine whether ata given time the mobile device is in a similar orientation of the samplematerial and capture guide; at a first time when the mobile device isnot in a similar orientation of the sample material and capture guide,determining to not capture the initial material sample image; and at asecond time when the mobile device is in a similar orientation of thesample material and capture guide, capturing the initial material sampleimage, without user input to capture the initial material sample imageat the second time. The second time may be after the first time.

Various pieces of metadata may be captured by the material datacollection system and associated with the sample material swatch area.For example, this may include an International Standards Organization(ISO) sensitivity when the initial material sample image was capturedand the second position for the first application provided alignmentindicator. The sample material swatch area may be used in rendering athree-dimensional model on a geometric form, where the renderedthree-dimensional model includes at least a portion of a surface of thethree-dimensional model including the sample material swatch area. Thesurface of the three-dimensional model may include at least two or morecopies of the sample material swatch area. Copies of the sample materialswatch area may be only partial copies of the sample material swatcharea. For example, smaller sample material swatch areas may need to beduplicated over ten times to be large enough to cover a surface of ageometric form. Edges of the geometry may not require an entire copy ofthe sample material swatch area and the material data collection systemmay identify and adjust the size of the copy accordingly.

In various implementations, transforming the initial material sampleimage, may be based upon a bounded polygon formed using a set ofapplication provided alignment indicators, to render an alignmentcorrected material sample image. The set of alignment indicators mayinclude four alignment indicators and the bounded polygon includes anirregular polygon. The set of alignment indicators included on thecapture guide may form a regular polygon. The transformed initialmaterial sample image may be used to identify a sample material swatcharea, where the sample material swatch area includes at least oneinstance of a pattern found in the initial material sample image andstored on the mobile device the sample material swatch area and theinitial material sample image. The sample material swatch area may beused to render a three-dimensional model on a geometric form, where therendered three-dimensional model includes at least a portion of asurface of the three-dimensional model including the sample materialswatch area.

In an implementation, the material data collection system includesproviding a sample material and a capture guide on top of the samplematerial. The capture guide may include a cut out portion that allows atleast a portion of the sample material to appear while the capture guideis on top of the sample material. For example, the sample material maybe placed on a tabletop, which is made of steel or other magneticmaterial. The capture guide may include magnets or other types offasteners that allow a user to move the capture guide or the samplematerial to prepare for capturing a digital image. For example, thesample material may be adjusted to eliminate or reduce creases beforethe digital image is captured. As another example, the capture guide maybe shifted or moved so that a relevant portion of the sample material isshown by the cut-out portion. For instance, an edge of the samplematerial is moved out of the cut-out portion, a defect of the samplematerial is moved out of the cut-out portion, or a complete instance ofa pattern of the sample material is moved into the cut-out portion. Thematerial data collection system may include capturing a digital image ofthe sample material and the capture guide on top of the sample material.The material data collection system may include transforming, based onthe capture guide captured in the digital image, the portion of thesample material appearing in the cut-out portion. The material datacollection system may support various transformations of the captureddigital image. For example, the material data collection system mayperform one or more of a color calibration of the digital image based ona color marker of the capture guide, a straightening of the digitalimage based on a straight line defined by the cut out portion of thecapture guide, or a deskewing of the digital image based on a polygonformed by corners of the cut out portion. The material data collectionsystem may determine a material swatch for the sample material, wherethe material swatch includes only the portion of the sample materialvisible in the cut-out portion.

In an implementation, the material data collection system includes colorcalibration including comparing, based on two different markers on thecapture guide that are of the same color, a color differential for thetwo different markers as captured in the digital image. The colordifferential may include a difference in luminance of one marker ascompared to the other marker. The material data collection system mayinclude determining, based on the color differential, a gradient tocompensate for the color differential of the digital image. For example,the gradient includes adjusting the color values according to the colordifferential within a preselected color space, such as a luminance valuein a YUV color space or other color spaces. The material data collectionsystem may include applying the gradient over the captured digital imageby adjusting color values of the captured digital image. The gradientmay be applied to points closest to one marker and gradually changeuntil points closer to the other marker, so that the color differentialhas been minimized or removed. More than two markers may be consideredwhen applying the gradient. Further, the process may be repeated by thematerial data collection system for markers of different colors.

In an implementation, the material data collection system includeschromaticity corrections. This may include comparing, based on twodifferent white point markers that are visible in the initial materialsample image, a chromaticity differential for the two different whitepoint markers; determining, based on the chromaticity differential, achromaticity gradient to compensate for the chromaticity differential;and applying the chromaticity gradient over the initial material sampleimage by adjusting chromaticity values. The material data collectionsystem may also perform both chromaticity and color correction. Thechromaticity correction may be performed before the color correction.

In an implementation, the material data collection system reduces anedge artifact of the sample material swatch area. The material datacollection system includes detecting a computer-readable indicationshown on the capture guide area of the initial material sample image;and determining a first direction for a sample material shown in thesample material area using the computer-readable indication. The firstdirection indicates a construction feature of the sample material. Forexample, the first direction may be a weft direction like the directionof yarns in a woven fabric (weft and warp). As another example, thedirection may be indicating natural features like neck, belly, and tailin animal skins.

In an implementation, the material data collection system includes arecolor feature. This may include converting the sample material swatcharea into a grayscale layer; receiving selection of a base color;applying the base color to at least a portion of the grayscale layer,where the applied base color matches the intensity of an original colorof the sample material swatch area in the base color.

In an implementation, the material data collection system includesmetadata storage of color information. The metadata may include colorsspaces different than the color space used in the captured image. Thismay include receiving selection of a first area from the sample materialswatch area or the initial material sample image and a selected colorspace; translating, from a color at the first area, to a matching coloridentifier in the selected color space; and storing as metadata with thesample material swatch area or the initial material sample image thematching color identifier.

In an implementation, the material data collection comprises a captureguide, where the capture guide includes: an opening on a top surface ofthe capture guide, where the opening is surrounded by edges of thecapture guide, and at least two alignment markers that are visible onthe top surface of the capture guide. The material data collectionincludes a mobile device, capturing in a digital image at least aportion of the top surface of the capture guide and a sample material,where the digital image is processed to produce a material swatch basedon the sample material and the at least two alignment markers. Thedigital image may include areas of the sample material appearing outsideof the opening. Further, the digital image may include objects otherthan the capture guide or the sample material.

In an implementation, a capture guide includes a rectangular framestructure including an exterior edge and an interior edge, where theinterior surrounds a perimeter of a rectangular first opening, a firstcorner of the rectangular frame structure, between the exterior edge andan interior edge, comprises a second opening, in a first direction at afirst distance from the second opening, a first alignment structure isformed on an upper surface of the rectangular frame structure, in asecond direction at a second distance from the second opening, a secondalignment structure is formed on the upper surface of the rectangularframe structure, the second direction is transverse to the firstdirection; and a metal piece, coupled to the second opening. The secondopening may be a circular shape. The first distance is the same as thesecond distance. The first alignment structure may include groovesformed on the upper surface of the rectangular frame structure and thegrooves outline approximately a rectangle or square shape. The secondalignment structure may include grooves formed on the upper surface ofthe rectangular frame structure, and the grooves outline a secondrectangular shape, and the first and second rectangular shapes have thesame area. The rectangular frame structure may include first ruledmarkings extending in the first direction and second ruled markingsextending in the first direction.

In an implementation, the material data collection includes a methodincluding receiving a digital image of a real-world material sample;analyzing the digital image to determine at least two marker areas foundon a capture guide included with the digital image; correcting, based onthe at least two marker areas, at least one of an alignment, color, orchromaticity of the digital image; and creating, after correcting thedigital image, a cropped version of the digital image including:removing the capture guide from the cropped version of the digitalimage, identifying at least one pattern of the real-world materialsample, and including the at least one pattern of the real-worldmaterial sample in the cropped version of the digital image. Includingthe at least one pattern may include receiving user input to determinethe at least one pattern. A tiled image of an initial cropped version ofthe digital image may be generated to determine whether edges of thepattern match. The at least two marker areas may include at least threedifferent colors and each of the at least two marker areas include thesame colors. For example, there may be two different colors used forcolor correction and one color for brightness correction. These colorsmay be known before the photo was captured. For example, the captureguide may be manufactured or constructed with these colors predefined,so that these are known colors before the digital image is captured.

Other objects, functionality, and advantages of the present inventionwill become apparent upon consideration of the following detaileddescription and the accompanying drawings, in which like referencedesignations represent like functionality throughout the figures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a simplified block diagram of a material data collectionsystem implemented in a distributed computing network connecting aserver and clients.

FIG. 2 shows a more detailed diagram of an exemplary client of thematerial data collection system.

FIG. 3 shows a system block diagram of a client computer system used toexecute application programs such as a web browser or performancesupport tools for the material data collection system.

FIGS. 4-5 show examples of mobile devices, which can be mobile clients.

FIG. 6 shows a system block diagram of a mobile device.

FIG. 7 shows a block relationship diagram of various components of thematerial data collection system.

FIGS. 8A-8B show a sample flow of data collection by the material datacollection system.

FIG. 9 shows an example of a sample material that is laid flat on atabletop.

FIG. 10 shows a schematic of an example of a capture guide that may beused with the material data collection system.

FIG. 11 shows an example of a particular capture guide that may be usedwith the material data collection system.

FIG. 12 shows an example of another capture guide in an isometric view.

FIG. 13 shows the example of the other capture guide in a close-up viewof a corner of the other capture guide.

FIG. 14 shows an example of a user holding a mobile device whileorienting the mobile device.

FIG. 15 shows an example screen capture of the material data collectionsystem in the calibration functionality.

FIG. 16 shows a screen capture of an image adjustment functionality.

FIG. 17 shows a sample screen capture of a crop verification preview.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 is a simplified block diagram of a distributed computer network100 incorporating an embodiment of the present invention. Computernetwork 100 includes a number of client systems 113, 116, and 119, and aserver system 122 coupled to a communication network 124 via a pluralityof communication links 128. Communication network 124 provides amechanism for allowing the various components of distributed network 100to communicate and exchange information with each other.

Communication network 124 may itself be comprised of many interconnectedcomputer systems and communication links. Communication links 128 may behardwire links, optical links, satellite or other wirelesscommunications links, wave propagation links, or any other mechanismsfor communication of information. Communication links 128 may be DSL,Cable, Ethernet or other hardwire links, passive or active opticallinks, 3G, 3.5G, 4G and other mobility, satellite or other wirelesscommunications links, wave propagation links, or any other mechanismsfor communication of information.

Various communication protocols may be used to facilitate communicationbetween the various systems shown in FIG. 1 . These communicationprotocols may include VLAN, MPLS, TCP/IP, Tunneling, HTTP protocols,wireless application protocol (WAP), vendor-specific protocols,customized protocols, and others. While in one embodiment, communicationnetwork 124 is the Internet, in other embodiments, communication network124 may be any suitable communication network including a local areanetwork (LAN), a wide area network (WAN), a wireless network, anintranet, a private network, a public network, a switched network, andcombinations of these, and the like.

Distributed computer network 100 in FIG. 1 is merely illustrative of anembodiment incorporating the present invention and does not limit thescope of the invention as recited in the claims. One of ordinary skillin the art would recognize other variations, modifications, andalternatives. For example, more than one server system 122 may beconnected to communication network 124. As another example, a number ofclient systems 113, 116, and 119 may be coupled to communication network124 via an access provider (not shown) or via some other server system.

Client systems 113, 116, and 119 typically request information from aserver system which provides the information. For this reason, serversystems typically have more computing and storage capacity than clientsystems. However, a particular computer system may act as both as aclient or a server depending on whether the computer system isrequesting or providing information. Additionally, although aspects ofthe invention have been described using a client-server environment, itshould be apparent that the invention may also be embodied in astand-alone computer system.

Server 122 is responsible for receiving information requests from clientsystems 113, 116, and 119, performing processing required to satisfy therequests, and for forwarding the results corresponding to the requestsback to the requesting client system. The processing required to satisfythe request may be performed by server system 122 or may alternativelybe delegated to other servers connected to communication network 124.

Client systems 113, 116, and 119 enable users to access and queryinformation stored by server system 122. In a specific embodiment, theclient systems can run as a standalone application such as a desktopapplication or mobile smartphone or tablet application. In anotherembodiment, a “web browser” application executing on a client systemenables users to select, access, retrieve, or query information storedby server system 122. Examples of web browsers include the InternetExplorer browser program provided by Microsoft Corporation, Firefoxbrowser provided by Mozilla, Chrome browser provided by Google, Safaribrowser provided by Apple, and others.

In a client-server environment, some resources (e.g., files, music,video, or data) are stored at the client while others are stored ordelivered from elsewhere in the network, such as a server, andaccessible via the network (e.g., the Internet). Therefore, the user'sdata can be stored in the network or “cloud.” For example, the user canwork on documents on a client device that are stored remotely on thecloud (e.g., server). Data on the client device can be synchronized withthe cloud.

FIG. 2 shows an exemplary client or server system of the presentinvention. In an embodiment, a user interfaces with the system through acomputer workstation system, such as shown in FIG. 2 . FIG. 2 shows acomputer system 201 that includes a monitor 203, screen 205, enclosure207 (may also be referred to as a system unit, cabinet, or case),keyboard or other human input device 209, and mouse or other pointingdevice 211. Mouse 211 may have one or more buttons such as mouse buttons213.

It should be understood that the present invention is not limited anycomputing device in a specific form factor (e.g., desktop computer formfactor), but can include all types of computing devices in various formfactors. A user can interface with any computing device, includingsmartphones, personal computers, laptops, electronic tablet devices,global positioning system (GPS) receivers, portable media players,personal digital assistants (PDAs), other network access devices, andother processing devices capable of receiving or transmitting data.

For example, in a specific implementation, the client device can be asmartphone or tablet device, such as the Apple iPhone (e.g., AppleiPhone 6), Apple iPad (e.g., Apple iPad or Apple iPad mini), Apple iPod(e.g., Apple iPod Touch), Samsung Galaxy product (e.g., Galaxy S seriesproduct or Galaxy Note series product), Google Nexus devices (e.g.,Google Nexus 6, Google Nexus 7, or Google Nexus 9), and Microsoftdevices (e.g., Microsoft Surface tablet). Typically, a smartphoneincludes a telephony portion (and associated radios) and a computerportion, which are accessible via a touch screen display.

There is nonvolatile memory to store data of the telephone portion(e.g., contacts and phone numbers) and the computer portion (e.g.,application programs including a browser, pictures, games, videos, andmusic). The smartphone typically includes a camera (e.g., front facingcamera or rear camera, or both) for taking pictures and video. Forexample, a smartphone or tablet can be used to take live video that canbe streamed to one or more other devices.

Enclosure 207 houses familiar computer components, some of which are notshown, such as a processor, memory, mass storage devices 217, and thelike. Mass storage devices 217 may include mass disk drives, floppydisks, magnetic disks, optical disks, magneto-optical disks, fixeddisks, hard disks, CD-ROMs, recordable CDs, DVDs, recordable DVDs (e.g.,DVD-R, DVD+R, DVD-RW, DVD+RW, HD-DVD, or Blu-ray Disc), flash and othernonvolatile solid-state storage (e.g., USB flash drive or solid statedrive (SSD)), battery-backed-up volatile memory, tape storage, reader,and other similar media, and combinations of these.

A computer-implemented or computer-executable version or computerprogram product of the invention may be embodied using, stored on, orassociated with computer-readable medium. A computer-readable medium mayinclude any medium that participates in providing instructions to one ormore processors for execution. Such a medium may take many formsincluding, but not limited to, nonvolatile, volatile, and transmissionmedia. Nonvolatile media includes, for example, flash memory, or opticalor magnetic disks. Volatile media includes static or dynamic memory,such as cache memory or RAM. Transmission media includes coaxial cables,copper wire, fiber optic lines, and wires arranged in a bus.Transmission media can also take the form of electromagnetic, radiofrequency, acoustic, or light waves, such as those generated duringradio wave and infrared data communications.

For example, a binary, machine-executable version, of the software ofthe present invention may be stored or reside in RAM or cache memory, oron mass storage device 217. The source code of the software of thepresent invention may also be stored or reside on mass storage device217 (e.g., hard disk, magnetic disk, tape, or CD-ROM). As a furtherexample, code of the invention may be transmitted via wires, radiowaves, or through a network such as the Internet.

FIG. 3 shows a system block diagram of computer system 201 used toexecute the software of the present invention. As in FIG. 2 , computersystem 201 includes monitor 203, keyboard 209, and mass storage devices217. Computer system 501 further includes subsystems such as centralprocessor 302, system memory 304, input/output (I/O) controller 306,display adapter 308, serial or universal serial bus (USB) port 312,network interface 318, and speaker 320. The invention may also be usedwith computer systems with additional or fewer subsystems. For example,a computer system could include more than one processor 302 (i.e., amultiprocessor system) or a system may include a cache memory.

Arrows such as 322 represent the system bus architecture of computersystem 201. However, these arrows are illustrative of anyinterconnection scheme serving to link the subsystems. For example,speaker 320 could be connected to the other subsystems through a port orhave an internal direct connection to central processor 302. Theprocessor may include multiple processors or a multicore processor,which may permit parallel processing of information. Computer system 201shown in FIG. 2 is but an example of a computer system suitable for usewith the present invention. Other configurations of subsystems suitablefor use with the present invention will be readily apparent to one ofordinary skill in the art.

Computer software products may be written in any of various suitableprogramming languages, such as C, C++, C#, Pascal, Fortran, Perl, Matlab(from MathWorks, www.mathworks.com), SAS, SPSS, JavaScript, AJAX, Java,Python, Erlang, and Ruby on Rails. The computer software product may bean independent application with data input and data display modules.Alternatively, the computer software products may be classes that may beinstantiated as distributed objects. The computer software products mayalso be component software such as Java Beans (from Oracle Corporation)or Enterprise Java Beans (EJB from Oracle Corporation).

An operating system for the system may be one of the Microsoft Windows®family of systems (e.g., Windows 95, 98, Me, Windows NT, Windows 2000,Windows XP, Windows XP x64 Edition, Windows Vista, Windows 7, Windows 8,Windows 10, Windows CE, Windows Mobile, Windows RT), Symbian OS, Tizen,Linux, HP-UX, UNIX, Sun OS, Solaris, Mac OS X, Apple iOS, Android, AlphaOS, AIX, IRIX32, or IRIX64. Other operating systems may be used.Microsoft Windows is a trademark of Microsoft Corporation.

Furthermore, the computer may be connected to a network and mayinterface to other computers using this network. The network may be anintranet, internet, or the Internet, among others. The network may be awired network (e.g., using copper), telephone network, packet network,an optical network (e.g., using optical fiber), or a wireless network,or any combination of these. For example, data and other information maybe passed between the computer and components (or steps) of a system ofthe invention using a wireless network using a protocol such as Wi-Fi(IEEE standards 802.11, 802.11a, 802.11b, 802.11e, 802.11g, 802.11i,802.11n, 802.11ac, and 802.11ad, just to name a few examples), nearfield communication (NFC), radio-frequency identification (RFID), mobileor cellular wireless (e.g., 2G, 3G, 4G, 3GPP LTE, WiMAX, LTE, LTEAdvanced, Flash-OFDM, HIPERMAN, iBurst, EDGE Evolution, UMTS, UMTS-TDD,1×RDD, and EV-DO). For example, signals from a computer may betransferred, at least in part, wirelessly to components or othercomputers.

In an embodiment, with a web browser executing on a computer workstationsystem, a user accesses a system on the World Wide Web (WWW) through anetwork such as the Internet. The web browser is used to download webpages or other content in various formats including HTML, XML, text,PDF, and postscript, and may be used to upload information to otherparts of the system. The web browser may use uniform resourceidentifiers (URLs) to identify resources on the web and hypertexttransfer protocol (HTTP) in transferring files on the web.

In other implementations, the user accesses the system through either orboth of native and nonnative applications. Native applications arelocally installed on the particular computing system and are specific tothe operating system or one or more hardware devices of that computingsystem, or a combination of these. These applications (which aresometimes also referred to as “apps”) can be updated (e.g.,periodically) via a direct internet upgrade patching mechanism orthrough an applications store (e.g., Apple iTunes and App store, GooglePlay store, Windows Phone store, and Blackberry App World store).

The system can run in platform-independent, nonnative applications. Forexample, client can access the system through a web application from oneor more servers using a network connection with the server or serversand load the web application in a web browser. For example, a webapplication can be downloaded from an application server over theInternet by a web browser. Nonnative applications can also be obtainedfrom other sources, such as a disk. In an implementation, the materialdata collection system uses the Open Graphics Library (OpenGL)application programming interface for rendering 2D and 3D vectorgraphics.

FIGS. 4-5 show examples of mobile devices, which can be mobile clients.Mobile devices are specific implementations of a computer, such asdescribed above. FIG. 4 shows a smartphone device 401, and FIG. 5 showsa tablet device 501. Some examples of smartphones include the AppleiPhone, Samsung Galaxy, and Google Nexus family of devices. Someexamples of tablet devices include the Apple iPad, Samsung Galaxy Tab,and Google Nexus family of devices.

Smartphone 401 has an enclosure that includes a screen 403, button 409,speaker 411, camera 413, and proximity sensor 435. The screen can be atouch screen that detects and accepts input from finger touch or astylus. The technology of the touch screen can be a resistive,capacitive, infrared grid, optical imaging, or pressure-sensitive,dispersive signal, acoustic pulse recognition, or others. The touchscreen is screen and a user input device interface that acts as a mouseand keyboard of a computer.

Button 409 is sometimes referred to as a home button and is used to exita program and return the user to the home screen. The phone may alsoinclude other buttons (not shown) such as volume buttons and on-offbutton on a side. The proximity detector can detect a user's face isclose to the phone, and can disable the phone screen and its touchsensor, so that there will be no false inputs from the user's face beingnext to screen when talking.

Tablet 501 is similar to a smartphone. Tablet 501 has an enclosure thatincludes a screen 503, button 509, and camera 513. Typically the screen(e.g., touch screen) of a tablet is larger than a smartphone, usually 7,8, 9, 10, 12, 13, or more inches (measured diagonally).

FIG. 6 shows a system block diagram of mobile device 601 used to executethe software of the present invention. This block diagram isrepresentative of the components of smartphone or tablet device. Themobile device system includes a screen 603 (e.g., touch screen), buttons609, speaker 611, camera 613, motion sensor 615, light sensor 617,microphone 619, indicator light 621, and external port 623 (e.g., USBport or Apple Lightning port). These components can communicate witheach other via a bus 625.

The system includes wireless components such as a mobile networkconnection 627 (e.g., mobile telephone or mobile data), Wi-Fi 629,Bluetooth 631, GPS 633 (e.g., detect GPS positioning), other sensors 635such as a proximity sensor, CPU 637, RAM memory 639, storage 641 (e.g.nonvolatile memory), and battery 643 (lithium ion or lithium polymercell). The battery supplies power to the electronic components and isrechargeable, which allows the system to be mobile.

The following is a high-level flow of the material data collectionsystem when used to capture material data. The flow includes a series ofsteps, however various embodiments of the material data collectionsystem may include greater or fewer steps than shown by the followingflow:

1. Alignment. The material data collection system includes an initialmaterial sample image. For example, the initial material sample imagemay be captured using a smart phone that includes a digital camera. Thedigital camera captures the initial material sample image with a samplematerial area and a capture guide appearing in the initial materialsample image. The capture guide may include one or more alignmentindicators, used to perform alignment correction in the material datacollection system. Any number of alignment indicators may be used by thematerial data collection system. In an implementation, the material datacollection system uses four alignment indicators so that a rectangle orsquare may be defined using the alignment indicators. Alternateimplementations of the material data collection system may use onealignment indicator (to define a circular cut-out portion), twoalignment indicators (to indicate diagonal matching corners for acut-out portion), five alignment indicators, or more.

In an implementation, the alignment indicator may be a visual indicationthat may be computer readable and recognized automatically by thematerial data collection system. For example, the visual indication maybe shown in the initial sample image and recognized by the material datacollection system using optical recognition techniques. Some examples ofvisual indications include a quick response (QR) code, a symbol (e.g.,“plus” sign, cross, dot, circle, or any other symbol), matrix barcode,two-dimensional barcode, linear barcode, or any other type of visualindicator. Alternate embodiments of the alignment indicator may includea non-visual indicator that may be captured and computer recognized. Forexample, electromagnetic, heat, metallic, or other non-visual indicatormay be used. Some examples of electromagnetic indicators includenear-field communications (NFC) and radio-frequency identification(RFID). The material data collection system may process the alignmentindicator to perform alignment adjustment. For example, the materialdata collection system will automatically recognize the alignmentindicator from the initial material sample image, without user inputspecifying where the alignment indicator is shown on the initialmaterial sample image. The alignment indicator may define a bounded areaof the initial material sample image where the sample material appears.The bounded area may correspond to any shape, such as a circle or apolygon with any number of edges (e.g., 3, 4, 5, 6, 7, or more edges).One or more alignment indicators may be used to define the bounded area.In an implementation, the alignment indicator includes a QR code. Acorner of the QR code may correspond to a corner defining an edge of thebounded area. For example, if the bounded area corresponds to arectangle or square, for an alignment indicator corresponding to a topright corner of the bounded area a bottom left corner of the QR code mayindicate where the top right corner of the bounded area should be to thematerial data collection system, for an alignment indicatorcorresponding to a top left corner of the bounded area a bottom rightcorner of the QR code may indicate where the top left corner of thebounded area should be to the material data collection system, for analignment indicator corresponding to a bottom right corner of thebounded area a top left corner of the QR code may indicate where thebottom right corner of the bounded area should be to the material datacollection system, and for an alignment indicator corresponding to abottom left corner of the bounded area a top right corner of the QR codemay indicate where the bottom left corner of the bounded area should beto the material data collection system. The QR code may include computerreadable information specifying which corner each respective QR belongs(e.g., the bottom left QR code indicates that the bottom left QR codeforms a bottom left corner for a bounded area).

2. Measurement. The initial sample image includes a measurement scalevisible on the capture guide. The measurement scale may correspond toany measurement unit, such as meters, inches, centimeters, or any othermeasurement unit. The measurement scale may be computer readable, sothat the material data collection system can understand the dimensionsof the sample area shown by the initial material sample image.

3. Lighting. The initial sample image includes color markers to performlighting correction. The material data collection system may use one ormore different color markers to compare and contrast the shown colors toperform the lighting corrections.

4. Rotation. The initial sample image includes orientation indicators.Depending on a type of material of the sample material, the samplematerial may indicate a direction of how the material should be orientedor a bias direction. For example, woven fabrics may have differentdirections such as a weft and warp, while other fabric types may haveother orientations. A selvage edge may also be indicated for the initialsample image. For example, for a woven fabric, selvage may run parallelto a warp of the fabric. The selvage may indicate an edge of the initialfabric that is self-finished.

FIG. 7 shows a block relationship diagram of various components of thematerial data collection system. The material data collection system maybe divided into three different portions, such as a mobile deviceportion 702, a product lifecycle management (PLM) system portion 704,and a sample material portion 705. The mobile device category portion702 includes a mobile device 706 that may be communicatively coupled toa product lifecycle management system 708 included in the productlifecycle management (PLM) system portion 704. The communication may bemade through any suitable means, such as a network connection over theInternet, an intranet, BLUETOOTH® network, or other network. The PLMsystem 708 may be any of a variety of available PLM systems. A specificimplementation of the PLM system includes PLM software available fromCentric Software, Inc. All public published content by Centric Software,Inc., including Web pages available at www.centricsoftware.com, to thefiling date of this patent application is incorporated by referencealong with all other cited references in this application. Thispublished content includes Web site pages, user guides and manuals,white papers, and other on-line and paper publications anddocumentation.

The mobile device 702 and the PLM system 704 may include varioushardware and software components. For example, the mobile device 706 mayinclude a touch screen 710 that allows a user to provide information tothe mobile device 706. The touch screen 710 may support many differenttypes of touch input, such as a selection, quick press, long press,force press, swipe, drag, or other touch input. The mobile device 706may include a data store 712. The data store 712 may be configured toselectively upload information from the data store 712, withoutadditional user input, to the PLM system 708. An application 714 may beinstalled onto the mobile device 706 to provide various functionality ofthe material data collection system. For example, the application 714may access through application programming interfaces provided by themobile device 706 access to various components of the mobile device 706,such as an orientation sensor 718. The orientation sensor 718 mayinclude a three-axis accelerometer. Three-axis indicates that theorientation sensor 718 may measure acceleration forces in threedimensions, so that whether the mobile device 706 is tilted or otherwisemoved, the mobile device 706 may capture and quantify the movement. Theorientation sensor 718 may be adapted to capture information such as theyaw, pitch, or roll of the mobile device 706. The mobile device 706 mayinclude a camera 720.

The sample material portion 705 may include a capture guide 724, asample material 726, and a tabletop 728. As explained in greater detailelsewhere in this application, the capture guide 724 is placed on top ofthe sample material 726. The tabletop 728 may be any flat surface,allowing the capture guide 724 and the sample material 726 to rest flatwhile a digital image is captured. The mobile device 706 may becalibrated, using the orientation sensor 718, to determine anorientation of the tabletop 728. This may be used to properly align theposition of the camera 720, while a digital image is being captured. Thecamera 720 may capture in a digital image the capture guide 724 and thesample material 726. Optionally, the light 722, such as a camera flash,may illuminate the capture guide 724 and the sample material 726 whenthe digital image is being captured.

Once a digital image has been captured, a material record 730 is createdfor the digital image. The material record 730 may include metadata 738of a sample material and the captured digital image 732. The captureddigital image 732 may include a cropped photo 734 and an uncropped photo736.

The PLM system portion 704 includes the PLM system 708, which storesmaterial records from the mobile device 706 in a data store 740. Thedata store 740 may include the metadata from material records, imagedata 748, including a cropped photo 750 and an uncropped photo 752, in afile store 746.

Some specific flows of the invention are presented below, but it shouldbe understood that the invention is not limited to the specific flowsand steps presented. A flow of the invention may have additional steps(not necessarily described in this application), different steps whichreplace some of the steps presented, fewer steps or a subset of thesteps presented, or steps in a different order than presented, or anycombination of these. Further, the steps in other implementations of theinvention may not be exactly the same as the steps presented and may bemodified or altered as appropriate for a particular application.

FIGS. 8A-8B show a sample flow of data collection by the material datacollection system. In a step 802, a material is laid on a tabletop.Surfaces other than a tabletop may also be used, as long as a user maylay the material on the surface while capturing data about the material.For example, the material may be laid on the ground, a book, pad, orother flat surfaces available. FIG. 9 shows an example of a samplematerial 902 that is laid flat on a tabletop 904, according to anembodiment of the material data collection system. The sample material902 is capable of being flattened or at least substantially flattened,such as a sample of fabric. One or more areas of the material may becaptured by the material data collection system, such as where there aremore than one instance of a pattern or design present on the fabric thatneeds to be captured.

In an implementation, the material data collection system may be used tocapture data about any material that is capable of being flattened orsubstantially flattened. This may allow the material data collectionsystem to fully capture details of the material, without shadows orother artifacts that may appear when capturing an object with irregularindentations or levels. Alternate embodiments of the material datacollection system may include irregularly shaped materials.

In a step 804, the material data collection system includes a captureguide placed on top of the material. The capture guide is placed over amaterial to assist properly orient the material data collection systemto capture material data. The capture guide may include indicators onthe capture guide capable of both human and computer recognition for usein orienting captured data. The capture guide may be positioned so thata cut out portion of the capture guide includes an area of interest ofthe material. For example, it may be a representative area of the typesof designs or elements of the material.

The capture guide may be of various sizes and shapes. In animplementation, the capture guide should be large enough to capture anentire pattern of a sample material within the cut-out window. Further,the capture guide may conform to various shapes and sizes of knownobjects that increase its portability or storage options. For example,the capture guide may conform to the letter (e.g., 215.9 by 279.4millimeters) or A4 (e.g., 210 by 297 millimeters) paper sizes. This mayallow users of the capture guide to include the capture guide inbinders, folders, bags, pockets, or other carrying options that arealready available.

In an implementation, the capture guide includes various thicknesses. Asan example, the capture guide may be of 0.5, 1, 2, 3, 4, 5 millimetersof thickness, or other thickness. The capture guide should be ofsufficient thickness so as to not be brittle and withstand numeroususes. However, the capture guide should not be so thick as to cause ashadow to appear when an image of it is captured by the material datacollection system. For example, when capturing a digital image of thecapture guide, the capture guide should be sufficiently thin so that theedges of the capture guide do not introduce shadows onto the samplematerial. A shadow may obscure or unnecessarily darken a portion of thesample material.

In an implementation, the material data collection system includes thedimensions of a capture guide, before it is included in a captureddigital image. This allows the material data collection system tocompute, with or without printed measurements on the capture guidesurface itself, the size of a cut out section of the capture guide. Forexample, if a size of the cut-out section is known, the material datacollection system may determine how much of the sample material appears.

FIG. 10 shows a schematic of an example of a capture guide that may beused with the material data collection system, in an embodiment.Alternate embodiments of the capture guide that may be used with thematerial data collection system may include more, fewer, or the sameelements as shown on FIG. 10 . The capture guide includes four alignmentindicators 102 that orient data captured when using the capture guide.The four alignment indicators 1002 may form corners of a cut out portion1004. The cut-out portion 1004 indicates where a material will bevisible when the capture guide is in use. The cut-out portion 1004 maybe of any shape, such as square, circular, rectangular, anysided-polygon, or other shape. This may depend on the number ofalignment indicators for any given capture guide. The dimensions of thecut-out portion 1004 may be stored by the material data collectionsystem as metadata associated with data captured from a sample. One ormore of the alignment indicator may be both human operator readable(e.g., a person may view the markings on the capture guide to orient thematerial) or machine readable (e.g., a mobile device using a sensor,such as a camera, may identify the markings). The capture guide may alsoinclude one or more measurement scales 1006. The capture guide shown inFIG. 10 includes an x-axis and a y-axis measurement scale. Themeasurement scale may use one or more of any appropriate measurements,such as inches, centimeters, meters, feet, millimeters, or othermeasurements.

The example of the capture may include orientation indicators andorientation labels 1008. The orientation indicators and orientationlabels 1008 may be used to indicate material specific characteristics ofa sample being stored. For example, the orientation labels may provide aprinted designation that may be read by a user to understand whetherweft or warp is in a certain direction. This may assist the user whenpreparing the capture guide to be placed on a sample material, so thatthe capture guide is placed in the correct direction. The orientationindicators may include computer-readable indication whether weft or warpis in a certain direction. The orientation indicators may be readable bythe material data collection system, without user input. In anembodiment, the orientation indicators and orientation labels 1008 maybe combined into a format that is readable by both a human and computer.

In an implementation, a capture guide may also be adapted tomaterial-specific characteristics that the capture guide is used with.Where the material data collection system is used with fabric or similarmaterials, the capture guide may include indications showing how toproperly orient the capture guide depending on characteristics of thefabric.

FIG. 11 shows an example of a capture guide that may be used with thematerial data collection system, in an embodiment. The capture guide maybe of any construction that facilitates the material data collectionsystem in capturing material data from a material. The other particularcapture guide may be constructed of acrylic, plastic, paper, cardboard,or any material that allows the other particular capture guide to beplaced upon a material and observed by the material data collectionsystem. For example, when made of paper, a cut out portion 1102 may beincluded, which indicates where to cut the capture guide for use withsample material.

The other particular capture guide of FIG. 11 includes a vertical axis1104 indicating “warp” and a horizontal axis 1106 indicating “weft.”These may indicate a bias direction for a fabric being used with thecapture guide. When fabric is constructed or woven, the weft (or woof)is the thread or yarn which is drawn through and insertedover-and-under, the lengthwise warp yarns that are held in tension on aframe or loom to create cloth. Warp is the lengthwise or longitudinalthread in a roll, while weft is the transverse thread. Proper alignmentof the warp and weft when used with the capture guide allows aconsistent orientation of data (e.g., digital image data) captured fromsamples. Alternate embodiments of the capture guide may also be adaptedfor use with other materials. For example, if the example material is areal or fake animal hide, it is important to note the grain or “spine”direction of the sample.

In an embodiment, a capture guide may be colored in a color adapted foruse with digital image processing techniques. For example, other thanmarkers indicating relevant colors located on the capture guide asexplained in greater detail elsewhere, the capture guide may be paintedin a digital green. This allows the mobile device to easily identifywhich areas of a captured digital image are part of the capture guideand which may be an area showing a sample material.

Returning to the schematic of an example of the capture guide of FIG. 10, the capture guide may include one or more magnets to properly positionthe capture guide on the tabletop. For example, the capture guide mayinclude embedded magnets 1010. While the capture guide is beingpositioned on the tabletop with the material, the magnets may hold intoplace the capture guide while still allowing a user to make adjustmentsto the positioning. As another example, the tabletop may include afiling cabinet, such as a top of the filing cabinet. The filing cabinetmay be constructed with metal, allowing the magnets embedded with thecapture guide to adhere to the filing cabinet using magnetic force.Alternate embodiments of the material data collection system may includea tabletop or capture guide using clips, binders, straps, fasteners,VELCRO®, or other methods to maintain a proper position of the captureguide. FIG. 12 shows an example of a capture guide in an isometric view.FIG. 13 shows the example of the capture guide in a close-up view of acorner of the capture guide. The capture guide includes four holes 1202that designate locations of the capture guide where a magnet may beplaced.

Returning to FIG. 10 , the schematic of the example of the capture guideincludes four marker portions 1012. The marker portions 1012 include twodifferent colors in this embodiment, such as a black marker portion anda white marker portion. The marker portions 1012 are also shown on FIG.13 as marker portions 1304. The marker portion 1304 may include a blackmarker portion below the magnet 1302 and a white marker portion to theleft of the magnet 1302. As explained in greater detail elsewhere, themarkers 1012 may be used by the material data collection system toprovide color or lighting calibration features.

Returning to FIG. 8A, in a step 806, an application on a mobile deviceis opened. The application may be a computer program running on thedevice that is used by a user to capture digital images. The applicationmay be downloaded from an application store available from the mobiledevice manufacturer, or other sources where applications may bedownloaded and installed on the mobile device. The mobile device may beany of a smartphone, tablet device, laptop, or other types of computingdevice. In an implementation, the mobile device includes differentcomponents such as a camera, a light, one or more orientation sensors, atouch screen, a data store, and a file store. Alternate embodiments mayinclude one or more of the components separate from the mobile device,but communicately coupled to the mobile device to provide data from thecomponent to the mobile device. For example, a digital single-lensreflex camera (DSLR) or mirrorless camera may provide digital images andassociated information to the mobile device for use.

In a step 808, the mobile device is laid upon the tabletop. In a step810, the material data collection system includes a calibrationfunctionality to record the tabletop orientation. This feature allowsthe mobile device to capture the orientation of the tabletop. The mobiledevice records the orientation of the table to calibrate itself toeliminate the need for a fixture to hold the device. For example, atabletop may be slightly tilted (e.g., one leg of the tabletop isshorter, the floor which the legs of the tabletop stand upon is uneven).This allows the material data collection system to capture high qualitydata about a material by noting the plane of the tabletop andpotentially making adjustments based on this information. As anotherexample, instead of a setup that fixes the position of the mobile devicefor image capture such as a tripod, the mobile device may be held by auser's hand. The user's hand may slightly shake or be tilted in one ormore directions. With the tabletop orientation information, the materialdata collection system may perform alignment adjustments on an image.

For example, the calibration functionality includes access to one ormore sensors of the mobile device. The mobile device uses the sensors tocapture the orientation of the tabletop or other surface that the samplematerial may be placed upon. In an implementation, the material datacollection system may use one or more accelerometers included with themobile device to capture orientation information used by the materialdata collection system. For example, when placing the mobile device onthe tabletop, the material data collection system may access informationfrom one, two, three, or more accelerometers built into the mobiledevice.

In a step 812, the mobile device is lifted from the tabletop surface. Ina step 814, the application enters into an image capture functionalityof the material data collection system. In an implementation, the imagecapture functionality uses the mobile device camera, a light, andorientation sensors to capture various pieces of information whilecapturing an image of the material.

In a step 816, the material data collection system includes orienting ormoving the mobile device while displaying an application image preview.FIG. 14 shows an example of a user holding a mobile device whileorienting the mobile device. The user may hold the mobile device 1402over a capture guide or material sample 1404 while performing this step.

In a step 818, the material data collection system includes orientingthe mobile device to align orientation indicators provided by theapplication. In an implementation, the material data collection systemmay determine the appropriate distance of the camera and the captureguide, when performing alignment. For example, capturing images in highquality may require a certain distance between a lens of the mobiledevice's camera and a capture guide. The appropriate distance may bedetermined by a variety of factors, such as the type of lens includedwith the mobile device (e.g., macro lens, telephoto, and standard),level of detail of the sample material, focusing capabilities of themobile device, or other.

In a step 820, the material data collection system includes aligning themobile device to orientation indicators. For example, the plane of thecamera and the plane of the tabletop should be aligned as closely aspossible to be in the same plane. The tabletop orientation may bedisplayed by the application, so that a user of the material datacollection system may attempt to correct the alignment if needed. Theapplication may include displaying a difference between the camera andthe tabletop orientation to assist proper alignment. The application mayprovide feedback to the user while showing the application imagepreview, to let the user know that, based on the application calibrationmode of step 810, how to tilt or otherwise position the mobile device sothat the orientation of the mobile device at any given moment iscompared to the captured table top orientation.

In a step 822, the material data collection system may include selectinga photo capture functionality provided by the application. This allowsthe material data collection system to capture a digital image of thematerial presented before the camera, along with associated metadata. Inan implementation, the material data collection system may include usinga lighting or flash functionality of the camera. For example, if thereis not enough lighting to properly illuminate the material with ambientlight when the photo is taken or to remove shadows or other lightingartifacts, the light or flash functionality may illuminate the samplematerial before or during data capture of the material. The lighting orflash functionality of the camera may be manually enabled or disabled bya user, depending on their subjective requirements or evaluations of thelighting conditions.

In an implementation, the photo capture functionality of the materialdata collection system may not engage immediately after input isreceived to capture a photo. The application may automatically capturethe photo when the orientation matches the accelerometer reading and thealignment indicators are all detected in the photo view. For example,the material data collection system may wait until the orientation ofthe tabletop and the mobile device are approximately the same. Thematerial data collection system may instead enter into an intermediatestage, to detect whether a threshold orientation alignment is met beforethe photo is captured. When the intermediate stage is entered, theapplication may automatically capture the photo, display a message tothe user to continue maintaining the orientation of the mobile device,or to make adjustments to the orientation before the photo isautomatically captured based on the received input.

In a step 823, the material data collection system determines corners ofa bounded area or a cut out section from an image captured by the photocapture functionality provided by the application. For example, as shownin FIG. 10 , a capture guide may include alignment indicators 1002.Depending on a shape of the cut-out section 1004, one or more alignmentindicators may be used by the material data collection system to formthe cut-out section 1004. FIG. 10 shows an embodiment of the captureguide where the cut-out section 1004 is a square. Four alignmentindicators 1002 are positioned at the corners of the cut-out section1004.

In an implementation, the alignment indicators include a computerreadable indicator. The application can automatically place the cornersusing the alignment indicators, which contain computer readableinformation regarding their location on the capture guide. The alignmentindicators may be in a shape or size that is known by the material datacollection system's application before an image of the capture guide iscaptured. The material data collection system may use an isolationalgorithm to identify the alignment indicators and where they arelocated. This means that the material data collection system mayinterpret, using an image of a capture guide including alignmentindicators, where the alignment indicators are positioned in the image,without user assistance. The computer readable indicator may be a QRcode. The QR code may include information indicating which corner itcorresponds to in the cut-out section. Additionally, corners of thealignment indicators may indicate where a corner of the cut-out sectionis located. For example, the alignment indicator may be located at a setdistance from the corner of the QR code. Also, the isolated alignmentindicators may be used with an edge detection algorithm to determinewhere the edges of the cut-out section are located. For example, if thecut-out section is rectangular or square, the material data collectionsystem may use straight lines to connect different corners of thecut-out section with each other to form a bounded area. In animplementation, the material data collection system uses Canny edgedetection. Using Canny edge detection, the material data collectionsystem may perform the steps of: (1) finding the intensity gradients ofan image; (2) applying non-maximum suppression to get rid of spuriousresponses to edge detection; (3) applying double threshold to determinepotential edges; and (4) tracking edge by hysteresis to finalize thedetection of edges by suppressing all the other edges that are weak andnot connected to strong edges.

In a step 824, the material data collection system allows the capturedphoto of the material to be viewed. In a step 826, the material datacollection system optionally includes an image adjustment functionality.The image adjustment functionality allows moving corners of thefour-sided shape to match alignment indicators of the capture guide. Forexample, the corners may be application provided alignment indicators,generated by the application, for use in correcting alignment. FIG. 15shows an example screen capture of the material data collection systemin the calibration functionality, in an embodiment. The screen captureshows a live view portion 1502 and a deskew option 1504. In the liveview portion 1502, the capture guide and material are shown. The liveview portion 1502 shows a first set of alignment indicators 1506included on the capture guide and captured by a camera of the mobiledevice. Additionally, the material data collection system includes inthe live view portion a second set of application provided alignmentindicators 1508 generated by the application. For each of the alignmentindicators in the first set, there is a matching alignment indicator inthe second set. A user of the material data collection system mayprovide appropriate input to adjust alignment indicators of the secondset to match those of the first set. For example, if the mobile deviceis a smartphone including a touch screen, the user may tap, slide, oruse other touch gestures to align the alignment indicators of the firstand second sets. When adjusting the second set of application providedalignment indicators, the live view portion 1502 may display a samplematerial captured in the photo 1510 and update the sample materialaccording to the adjustments at any given time. For example, by changingthe second set of application provided alignment indicators, a polygonformed by the second set of application provided alignment indicatorsmay change in shape. The sample material captured in the photo 1510 maybe updated according to the changes to the second set of applicationprovided alignment indicators.

In an embodiment, the material data collection system may include adeskewing feature. The deskewing feature may correct an image that isslanting too far in one direction, or one that is misaligned. Thedeskewing feature may be performed by the application, without inputfrom a user as to how to deskew a captured digital image. For example,the user may select the deskew button shown in FIG. 15 to automaticallydeskew. The deskew feature implemented by the application may usevarious deskewing methods. For example, the application may applyfour-point perspective transformation to the image. The deskew featuremay take four points and transform the image so that the four pointshave ninety-degree corners. In an implementation, the four points may bealignment indicators captured in an image.

The material data collection system may also pass a maximum width andheight. However, the material data collection system may need additionaluser input to correct a final aspect ratio is correct so that thematerial data collection system may vertically scale or horizontallyscale the image. In an implementation, the material data collectionsystem is aware of the size of a capture guide in a digital image. Withthe capture guide measurements, the material data collection system mayuse the alignment indicators to automatically position the four pointsto feed into the four-point perspective transformation algorithm. Withthe capture guide, the material data collection system knows the exactdimensions of the cutout portion, and the material data collectionsystem may use this information to automatically apply the vertical orhorizontal scaling to correct the image after the four-point perspectivetransformation is applied. Without the capture guide, a user may need tomanually move the points to the correct location before applying thefour-point perspective transformation.

FIG. 16 shows a screen capture of an image adjustment functionality, inan embodiment. For example, if a photo of the material was at aninappropriate distance or incorrect orientation, the user may move thecorners of the digital image to adjust the crop of the digital image. InFIG. 16 , the digital image is a square, but the material datacollection system may be adapted for use with any polygon shapes. Inthis example, a digital image has captured a sample material 1604 andmeasurement scales 1606. A user may adjust application providedalignment indicators 1602 to select the corners of the sample material.

In a step 828, the application transforms the digital image according tothe input provided. The application can use the features detected in thephoto together with the indicators on the capture guide to automaticallyapply transformations. Some examples of transformations possible includeperspective transformation, rotation transformation, fisheye projection,and affine transformation. For example, the application uses theinformation specified by the four circular points to perform one or moreof deskewing the digital image, rotating the image, scaling the image,or other adjustments. Further, the material data collection system mayuse positions of the application provided alignment indicators to removethe capture guide or calculate a transformation to remove lensaberration effects, such as lens warp or fisheye effect. In animplementation, the material data collection system applies a bumpdistortion transformation to the digital image to reverse the effects ofthe lens, like fisheye warping. With the capture guide, the materialdata collection system may use the edges of the capture guide to autoadjust the parameters of the transformation so that the pixels on theedges match a straight line between the cut-out section corners. Thematerial data collection system may also transform the pictures byremoving edges or corners of the digital images, for example to correctfor a pin cushion effect.

In a step 830, the material data collection system selects black markersfrom the capture guide in the capture digital image. The black markersmay contain computer readable information regarding their location onthe capture guide. Alternate implementations of the material datacollection system may include other markers, such as white markers,black markers, RYB, RYBGCM, or any combination of these.

In a step 832, the material data collection system selects calibrationmarkers from the capture guide in the capture digital image. In animplementation, the material data collection system includes black orwhite markers or both to perform chromaticity correction. The materialdata collection system may also include color markers in conjunctionwith black or white markers to perform color correction. The markers maycontain computer readable information regarding their location on thecapture guide. For example, markers may be paired so that each markerindicates its location as compared to one or more other markers on thecapture guide.

In a step 834, the material data collection system may performcorrections on the captured digital image. Embodiments of the materialdata collection system may perform color correction, chromaticitycorrection, or both using markers. Using the white and black markers,the application can apply a color correction to even the lighting andset the proper black and white points for the pixels. This may assistthe material data collection system in generating a “true” color imagefor the captured digital image, that does not include any artifactsintroduced by different lighting sources, intensities, or angles. Forexample, there may be overhead lighting, natural window lighting, shadowfrom the camera or camera user, and table lighting all captured in thesame digital image. The material data collection system may adjust thecaptured digital image, so that variations of lighting introduced by thedifferent lighting sources is corrected.

In the embodiment where a capture guide includes black or white markersto assist the material data collection system in color correction, theblack or white markers may be included on the capture guide, in an areathat does not obscure the sample. A black marker on the surface of thecapture guide is colored flat black or RGB (0, 0, 0). This is used bycolor correction logic to adjust the photo pixels in a way to make theblack marker a specific pixel color. A white marker on the surface ofthe capture guide may be flat white or RGB (255, 255, 255). This is usedby the color correction logic to adjust the photo pixels in a way tomake the white marker a specific pixel color. The material datacollection system may use the color correction logic to adjust thecolors of the image so that the white and black markers are all the samerespective pixel color. In an implementation, the color correction logicmay use Contrast Limited Adaptive Histogram Equalization (CLAHE).

The material data collection system may include analyzing markers foundon the capture guide, to perform color analysis and adjustments. Forexample, the color analysis may adjust brightness across an image sothat markers captured in the image are the same. This may makebrightness across the image even. If the material data collection systemdetermines that a captured digital image with a white marker located atone section of the capture guide is, in comparison to another whitemarker located at another section of the capture guide, that there islighting discrepancy between the two points. The white marker may appearto be brighter (e.g., higher luminescence) than the other white marker.The material data collection system may make adjustments to the captureddigital image to compensate for the difference. For example, thematerial data collection system may apply a gradient to make thecaptured digital image darker near the white marker and brighter nearthe other white marker. The material data collection system may considerany number of markers to determine adjustments need to a captureddigital image. For example, if a particular capture guide includes fourwhite markers and four black markers, the material data collectionsystem may consider all eight markers or any subset of the markers todetermine whether adjustments are necessary.

Alternate implementations may include markers with colors other thanwhite and black to perform color correction. For example, instead ofblack and white, the capture guide may include a marker to calibrate acaptured digital image in another color space, such as red-green-blue(RGB), ADOBE™ RGB, RYB, RYBGCM, or any other color space or subset of acolor space. This may allow the same or a similar color temperature tobe applied throughout the entire image.

In an implementation, the material data collection system may includeboth chromaticity correction and color correction for a single sample.The material data collection system may perform the chromaticitycorrection, before the color correction.

Returning to FIG. 8B, in a step 836, the material data collection systemmay select a crop region for repeat and preview. The application canautomatically select a cropped region for the repeat, and the user canmanually adjust the region as needed. This allows seamless stitching ofthe crop region when applied to a geometry. The material data collectionsystem may modify the crop region to prepare the cropped region forapplication on a geometry. For example, the material data collectionsystem may apply an edge blurring algorithm. In addition to performingalignment corrections when applying on a geometry, the material datacollection system modifies edges so that possible edge artifacts aresuppressed. That way, when a pattern from the cropped region isrepeated, a stitching algorithm may stitch the pattern and reconcile howedges of the pattern should properly look when put together.

In an implementation, the material data collection system may include anedge detection algorithm that identifies where an instance of a patternmay be found in a captured digital image. Alternate implementations ofthe material data collection system may include user identification ofwhere an instance of a pattern is found.

In a step 838, the material data collection system may adjust the edgesof the cropped region to blend edges of the repeated crop region. Eachcrop region may include at least one complete instance of a pattern ofthe sample material. FIG. 17 shows a sample screen capture of a cropverification preview. This may allow a user to verify whether thecropped version of a sample material was correctly done or needs to beadjusted. If the user determines the cropped version does not accuratelyreflect the material, the user may retry and create a new croppedregion. The crop verification preview may be displayed to a user, sothat the user may confirm that the cropped version appears accuratelywhen the cropped version is duplicated. In the example shown on FIG. 17, nine cropped areas 1702 are shown. Each cropped area may show the samecropped area. The cropped area may include patterns or designs, such ascircles 1704, 1706, 1708, and 1710. As shown in FIG. 17 , a user mayview the repeated crop area and confirm that the cropped area wascorrectly identified, since the circles 1704, 1706, 1708, and 1710correctly line up across multiple repeated cropped areas.

In a step 840, the material data collection system saves with thematerial data an uncropped version of the digital image. This mayinclude the digital image showing the capture guide. The capture guidemay include alignment indicators, labels, and measurement scales toassist a user that retrieves the material data in understanding theattributes of the material.

In a step 842, the material data collection system may save a croppedversion or swatch of the digital image of the material. For example, thematerial shown in the digital image as shown in the cut-out portion ofthe capture guide may be stored separately, to provide a cleaner view ofthe material without the capture guide.

In a step 844, the material data collection system allows saving thedigital image and associated metadata. In addition to the metadatadescribed elsewhere in this application, relevant metadata may include ascale of the digital image, camera settings used when the digital imagewas captured (e.g., f/stop, whether flash was on, location information,brightness adjustments, ISO levels).

In a step 846, the material data collection system applies the croppedversion of the sample material. For example, the cropped version of thesample material may correspond to a swatch sample of the fabric capturedin the digital image. The swatch sample may be repeated horizontally orvertically as many times as needed to create a virtual fabric of desiredsize as it would appear on a geometry. The virtual fabric should lookcontinuous, instead of a checkerboard effect where it is apparent thatthe virtual fabric is generated using stitched together sample swatches.Eventually the swatch sample can be applied to create images of clothingworn by people (e.g., showing how the fabric drapes, how the fabricwould appear in different lighting).

The material data collection system may include various geometries, suchas human forms, to apply the cropped version. The geometries may be usedto model the sample material. The geometries may include differentregions for different clothing items, such as a shirt region, a pantsregion, a dress region, or other regions. The cropped version isadjusted to conform to the contours of the geometry. The cropped versionmay be applied to one or more regions of the geometries and repeated oneor more times, so that it appears as if the model is dressed in thesample material. In an implementation, the geometries are placed in avirtual reality background, such as using a mock background or anaugmented reality background, so that a user may properly appreciate themodel in a real-life space.

In an implementation, a swatch or cropped version of the sample materialmay indicate how to combine two or more instances of the swatch.Multiple instances of the swatch may be stitched together depending ondimensions of a geometry the swatch is being applied on. Largergeometries may require more instances of a swatch than for smallergeometries. For example, a first edge of a swatch may indicate how toalign the first edge with a second edge that is opposite from the firstedge on the swatch, while a third edge of the swatch may indicate how toalign the third edge with a fourth edge that is opposite from the thirdedge on the swatch.

The material data collection system may also perform image scaling on aswatch. Instead of a manual process using photo editing software toensure the captured dimensions for a swatch are correct, the materialdata collection system may automatically adjust the swatch dimensions.For example, a capture guide has known dimensions for a cut-out portion.So, when the capture guide is used, the sample material that appears inthe cut-out portion should have the same or similar dimensions as theknown dimensions. Because of tilting or other image issues however, thedimensions shown in the image may be incorrect. For example, if acut-out portion is known to be 0.05 meters in width and 0.05 meters inheight but a sample material capture using the capture guide showsdimensions of 0.049 meters in width and 0.052 meters in height, thematerial data collection system may adjust the swatch dimensions. In theexample, the material data collection system may stretch the swatch inwidth and shrink the swatch in height.

In an implementation, the material data collection system uses acaptured image and, based on the captured image, creates a materialswatch using automated techniques. This means that, although options maybe provided by the material data collection system for a user to reviewand make adjustments to the material swatch created, once the userindicates to capture the image, the material data collection system mayperform the rest of the steps to create the material swatch.

In an implementation, the material data collection system allowsswapping or recoloring of colors in a captured image. For example, auser may like a material captured by the material data collectionsystem, but not a color the material is in. The material data collectionsystem allows the user to swap the color from the color found in theimage to one that they select, without needing to procure a new versionof the sample material in a new color. In an implementation, thematerial data collection system swaps colors using the following method:

(1) A captured image of a generated swatch or other image is retrievedby the material data collection system.

(2) The captured image is converted into grayscale or desaturated. Whilein grayscale, the captured image maintains the intensity information ofthe original captured image, but with one or more colors found in thecaptured image transformed into gray.

(3) The user may select one or more new colors in the material datacollection system to replace the one or more colors. The user may selectthe one or more new colors directly using a LAB value or from a palateof colors.

(4) The material data collection system applies the one or more newcolors, using the intensity information from the grayscale capturedimage, to change the one or more colors into the one or more new colors.For example, the material data collection system may use a multiplycommand. The multiply command goes through pixels of the captured imageand uses intensity information from the grayscale multiplied by the newcolor to swap colors. If there are two or more colors to be swapped,step (4) may be repeated as necessary.

In an implementation, the material data collection system uses adevice's digital camera and tilt sensor, for the features as describedabove. However, the material data collection system does not use thedevice's accelerometer or distance sensor. Further, the material datacollection system may not need to use autofocus camera distanceinformation, or augmented reality features provided by the device (i.e.,Augmented Reality such as ARKit in Apple Inc.'s iOS™, ARCore in Google'sAndroid®)The material data collection system may understand distances ofobjects captured in the image using only the photographed capture guide.

In an implementation, the material data collection system allowsmetadata storage of colors found in a captured image. This allows usersto make meaningful comparisons of different swatches and search fordifferent colors, even if the color identifiers used are from adifferent color space. The metadata may be searchable, so that users canretrieve different samples using color identifiers from different colorspaces. For example, pixels in the captured image may be represented inthe material data collection system in either red, green, and blue(RGB), cyan, magenta, yellow, and black (CMYB), or hex formats. However,designers or other users may be more familiar with other color spaces,such as PANTONE®, CSI's Color Library™, or CHROMA. The material datacollection system may convert the pixel color information into theircorresponding color in any of these color spaces and store theinformation as metadata associated with the captured image. For example,a pixel may include the color information RGB 0 35 156, HEX 00239C, orCMYK 988200. However, in the PANTONE color space, this color may beknown as Dark Blue C.

In an implementation, the material data collection system may generatethe color metadata automatically, without user input to specificallygenerate the metadata. For example, the metadata may be automaticallygenerated when a swatch is created or after color or brightnesscorrection has occurred. The material data collection system may alsogenerate the color metadata based on a user's input. For example, theuser may select an area of the captured image to convert the colorinformation in the area into a color identifier in a selected colorspace. In an implementation, the material data collection system mayprovide approximate conversions for colors. For example, some colorspaces may not include the entire spectrum of possible colors. Thematerial data collection system may attempt to find a closest match forthe selected area.

In an implementation, the material data collection system includes oneor more features to transform digital images. The table belowillustrates an example of various issues and techniques that may be usedby the material data collection system to resolve the issues. Thematerial data collection system may support or apply one, two, three, orany number of the features in various embodiments of the material datacollection system. Other digital image issues not listed below may alsobe corrected by the material data collection system.

TABLE Automated Process to Digital Image Issue How to Detect ResolveIssue Pin cushion, barrel, Detect distortion based Apply Brown-Conradyor mustache on a deviation from model distortion rectilinear projectionStraighten/deskew Analyze alignment Use one or more straight indicatorsedges from a cut out portion of a capture guide as a straight edge andtransform the digital image based upon the straight edge as captured inthe digital image. Misalignment Analyze alignment Determine locationsfor indicators alignment indicators. Overlay a digital image with theapplication provided alignment indicators. Optionally, allow adjustmentof positions for the application provided indicators. Transform digitalimage, based on location of the application provided alignmentindicators. Color/lighting Compare marker colors CLAHE correction (e.g.,black and white markers) Dimensions of Compare dimensions of Applystretching or swatch out of sample material area shrinking algorithm toproportion/image shown in an image to adjust one or both scaling knowndimensions of a height and width of cut out area. For swatch to knownsize of example, if a cut out area a cut out portion. is known to be acertain dimension, the sample material area should have similardimensions. Edge artifact for Edges of a swatch do not Provide croppedversion swatch properly match with or swatch in a crop another edge ofthe verification preview. swatch if the swatch is Allow adjustment tostitched. cropped version or swatch, if pattern or other features of asample material are misaligned. An edge artifact algorithm may also beapplied to prepare a swatch for stitching.

This description of the invention has been presented for the purposes ofillustration and description. It is not intended to be exhaustive or tolimit the invention to the precise form described, and manymodifications and variations are possible in light of the teachingabove. The embodiments were chosen and described in order to bestexplain the principles of the invention and its practical applications.This description will enable others skilled in the art to best utilizeand practice the invention in various embodiments and with variousmodifications as are suited to a particular use. The scope of theinvention is defined by the following claims.

The invention claimed is:
 1. A method comprising: capturing an initialsample image using an application executing on a mobile device, whereinthe initial sample image includes at least the following visible areas:a sample area, and a capture guide area including a capture guide and analignment indicator included as part of the capture guide; identifyingthe alignment indicator in the initial sample image; based on theidentified alignment indicator, transforming the initial sample image toobtain an alignment corrected sample image; in the alignment correctedsample image, identifying a sample swatch area, wherein the sampleswatch area includes at least one instance of a pattern found in theinitial sample image; and storing on the mobile device the sample swatcharea and the initial sample image.
 2. The method of claim 1 whereintransforming the initial sample image comprises determining a boundedarea where the sample area is shown in the initial sample image.
 3. Themethod of claim 2 wherein the bounded area corresponds to the sampleswatch area.
 4. The method of claim 1 wherein transforming the initialsample image comprises correcting a skew of the initial sample image. 5.The method of claim 1 wherein the identifying the alignment indicator inthe initial sample image is performed using an optical recognition ofthe alignment indicator.
 6. The method of claim 1 comprising: beforecapturing the initial sample image, determining via the mobile device anorientation of a sample and capture guide; and before capturing theinitial sample image, providing visual feedback indicating whether at agiven time the mobile device is in a similar orientation of the sampleand capture guide.
 7. The method of claim 6 wherein taking anorientation comprises using a three-axis accelerometer of the mobiledevice.
 8. The method of claim 1 comprising associating with the sampleswatch area metadata comprising an International Standards Organization(ISO) sensitivity when the initial sample image was captured.
 9. Themethod of claim 1 comprising: rendering a three-dimensional model on ageometric form, wherein the rendered three-dimensional model comprisesat least a portion of a surface of the three-dimensional model includingthe sample swatch area.
 10. The method of claim 9 wherein the geometricform comprises at least a portion of a human body.
 11. The method ofclaim 9 comprising creating the surface of the three-dimensional modelbased on repeating at least two or more copies of the sample swatcharea.
 12. The method of claim 1 wherein the capture guide area entirelyencloses at least a portion of the sample area in the initial sampleimage.
 13. The method of claim 1 comprising: before capturing theinitial sample image, taking an orientation of a sample and captureguide by the mobile device; before capturing the initial sample imageand after a user input to capture the initial sample image, accessing anaccelerometer of the mobile device to determine whether at a given timethe mobile device is in a similar orientation of the sample and captureguide; at a first time when the mobile device is not in a similarorientation of the sample and capture guide, determining to not capturethe initial sample image; and at a second time when the mobile device isin a similar orientation of the sample and capture guide, capturing theinitial sample image, without user input to capture the initial sampleimage at the second time.
 14. The method of claim 1 comprising:detecting a computer-readable indication shown on the capture guide areaof the initial sample image; and determining a first direction for asample shown in the sample area using the computer-readable indication.15. The method of claim 1 comprising: comparing, based on two differentwhite point markers that are visible in the initial sample image, achromaticity differential for the two different white point markers;determining, based on the chromaticity differential, a chromaticitygradient to compensate for the chromaticity differential; and applyingthe chromaticity gradient over the initial sample image by adjustingchromaticity values.
 16. The method of claim 1 comprising: comparing,based on two different color markers of a single color that are visiblein the initial sample image, a color differential for the two differentcolor markers; determining, based on the color differential, a colorgradient to compensate for the color differential; and applying thecolor gradient over the initial sample image by adjusting color values.17. The method of claim 1 comprising: comparing, based on two differentwhite point markers that are visible in the initial sample image, achromaticity differential for the two different white point markers;determining, based on the chromaticity differential, a chromaticitygradient to compensate for the chromaticity differential; applying thechromaticity gradient over the initial sample image by adjustingchromaticity values; comparing, based on two different color markers ofa single color that are visible in the initial sample image, a colordifferential for the two different color markers; determining, based onthe color differential, a color gradient to compensate for the colordifferential; and applying the color gradient over the initial sampleimage by adjusting color values.
 18. The method of claim 17 whereinapplying the chromaticity gradient is before applying the colorgradient.
 19. The method of claim 1 comprising: converting the sampleswatch area into a grayscale layer; receiving selection of a base color;applying a base color layer to at least a portion of the grayscalelayer, wherein the applied base color matches the intensity of anoriginal color of the sample swatch area.
 20. The method of claim 1comprising: receiving selection of a first area from the sample swatcharea or the initial sample image and a selected color space;translating, from a color at the first area, to a matching coloridentifier in the selected color space; and storing as metadata with thesample swatch area or the initial sample image the matching coloridentifier.